> 



PATENT ABSTRACTS OF JAPAN 
(ll)Publication number : 2000-324428 
(43)Date of publication of application : 24. 11.2000 



(5l)lnt.Cl. H04N 5/76 
H04N 7/32 



(21) Application number : 11-133937 (7l)Applicant : CANON INC 

(22) Date of filing : 14.05. 1999 (72)Inventor : ICHIHASHI NOBUHARU 



(54) DEVICE AND METHOD FOR DELAYED REPRODUCTION, AND STORING 
MEDIUM 

(57)Abstract: 

PROBLEM TO BE SOLVED: To realize catch-up reproduction corresponding to the kind 
of watched broadcasting in a system for simultaneously recording and reproducing 
digital television broadcasting. 

SOLUTION: This device is provided with an information supplying means 1 for 
supplying multiplexed data information of video, voice and other data sent by digital 
signals, a recording/reproducing means 9 for recording and reproducing a bit stream 
continuously outputted from the means 1, a data control means 5 for executing the 
input control and the output control of data to be recorded in and reproduced from the 
means 9, an operation monitoring means 10 for monitoring the motion of the video 
(voice) part of data to be recorded in the means 9 and a frame number deciding means 6 
for deciding the number of frames to be reproduced in a unit time from the means 9 



based on the monitoring result of the means 10. In the case of interrupting viewing of 
broadcasting for an optional period, it is possible to catch up with rear broadcasting by 
reproducing faster than the real broadcasting from the interrupted part. 
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* NOTICES * 
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damages caused by the use of this translation. 



1 .This document has been translated by computer. So the translation may not 

reflect the original precisely. 

2.**** shows the word which can not be translated. 

3.ln the drawings, any words are not translated. 

CLAIMS 
[Claim(s)] 

[Claim 1] The image sent with a digital signal, voice, and other information 
supply means to supply the multiplexing data information of data, The record 
playback means for recording the bit stream continuously outputted from the 
above-mentioned information supply means, and reproducing, A data control 
means to perform the entry-of-data control recorded on the above-mentioned 
record playback means, and the output control of the data reproduced from the 
above-mentioned record playback means, The delay regenerative apparatus 
characterized by providing a performance monitor means to supervise a motion 
of the image part of the data recorded on the above-mentioned record playback 
means, and a frame number decision means to determine the frame number 
reproduced from the above-mentioned record playback means to unit time 



amount based on the monitor result of the above-mentioned performance 
monitor means. 

[Claim 2] The monitor result of the above-mentioned performance monitor 
means is a delay regenerative apparatus according to claim 1 characterized by 
asking for the sum of the motion vector of a macro block, and generating based 
on the magnitude. 

[Claim 3] The monitor result of the above-mentioned performance monitor 
means is a delay regenerative apparatus according to claim 1 or 2 characterized 
by asking for the sum of the absolute value of the motion vector of a macro block, 
and generating based on the magnitude. 

[Claim 4] The above-mentioned macro block is a delay regenerative apparatus 
given in any 1 term of claims 1-3 which extract only from several places of the 
screen and are characterized by patternizing a motion of an image and 
considering as the output of the above-mentioned performance monitor means 
with the combination of the sum of a motion vector, and the sum of the absolute 
value of a motion vector. 

[Claim 5] Images, such as broadcast of a digital signal, voice, and other 
information supply means to supply the multiplexing data information of data, 



The record playback means for recording the bit stream continuously outputted 
from the above-mentioned information supply means, and reproducing, The 
entry-of-data control recorded on the above-mentioned record playback means, 
and a data control means to control the output of the data reproduced from the 
above-mentioned record playback means, The delay regenerative apparatus 
characterized by providing a sound-volume monitor means to supervise the 
sound volume of the voice part of the data recorded on the above-mentioned 
record playback means, and a frame number decision means to determine the 
frame number reproduced from the above-mentioned record playback means to 
unit time amount based on the monitor result of the above-mentioned 
sound-volume monitor means. 

[Claim 6] A delay regenerative apparatus given in any 1 term of claims 1-5 
characterized by providing the image storage means for compounding a display 
image, and an expansion composition means to develop and compound the 
frame of the frame number reproduced to unit time amount for the 
above-mentioned image storage means from the above-mentioned record 
playback means. 

[Claim 7] The image sent with a digital signal, voice, and the other information 



provisioning processes which supply the multiplexing data information of data, 
The record regeneration which records the bit stream continuously outputted by 
the above-mentioned information provisioning process on a record playback 
means, and is reproduced, The data control processing which performs the 
entry-of-data control recorded on the above-mentioned record playback means, 
and the output control of the data reproduced with the above-mentioned record 
playback means, The delay playback approach characterized by performing 
performance monitor processing which supervises a motion of the image part of 
the data recorded on the above-mentioned record playback means, and frame 
number decision processing in which the frame number reproduced to unit time 
amount with the above-mentioned record playback means is determined based 
on the output of the above-mentioned performance monitor processing. 
[Claim 8] The output of the above-mentioned performance monitor processing is 
the delay playback approach according to claim 7 characterized by asking for 
the sum of the motion vector of a macro block, and generating based on the 
magnitude. 

[Claim 9] The output of the above-mentioned performance monitor processing is 
the delay playback approach according to claim 7 or 8 characterized by asking 



for the sum of the absolute value of the motion vector of a macro block, and 
generating based on the magnitude. 

[Claim 10] The above-mentioned macro block is the delay playback approach 
given in any 1 term of claims 7-9 which extract only from several places of the 
screen and are characterized by patternizing a motion of an image and 
considering as the output of the above-mentioned performance monitor 
processing with the combination of the sum of a motion vector, and the sum of 
the absolute value of a motion vector. 

[Claim 11] Images, such as broadcast of a digital signal, voice, and the other 
information provisioning processes that supply the multiplexing data information 
of data, The record regeneration which records the bit stream continuously 
outputted by the above-mentioned information provisioning process on a record 
playback means, and is reproduced, The entry-of-data control recorded on the 
above-mentioned record playback means, and the data control processing 
which controls the output of the data reproduced with the above-mentioned 
record playback means, The delay playback approach characterized by 
performing sound-volume monitor processing which supervises the sound 
volume of the voice part of the data recorded on the above-mentioned record 



playback means, and frame number decision processing in which the frame 
number reproduced to unit time amount with the above-mentioned record 
playback means is determined based on the output of the above-mentioned 
sound-volume monitor processing. 

[Claim 12] The delay playback approach given in any 1 term of claims 7-1 1 
characterized by performing image storage processing for compounding a 
display image, and expansion composition processing which develops and 
compounds the frame of the frame number reproduced to unit time amount with 
the above-mentioned record playback means to the above-mentioned image 
storage processing. 

[Claim 13] The storage characterized by storing from a computer the program for 
operating a computer as each means to constitute a delay regenerative 
apparatus according to claim 1 to 6 possible [ read-out ]. 
[Claim 14] The storage characterized by storing the program for making a 
computer perform the procedure of the delay playback approach of a publication 
in any 1 term of claims 7-12 possible [ read-out ] from a computer. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] It reproduces from live data at high speed, and this 
invention relates to the technique which catches up with live data, after it records 
single string data on a time-axis temporarily and the time amount of arbitration 



passes especially about a delay regenerative apparatus, the delay playback 
approach, and a storage. It uses in order to catch up with real broadcast by 
reproducing from real broadcast at high speed especially from the place which 
the broadcast data (MPEG 2 transport stream) of digital broadcasting are an 
object, interrupted the time amount of arbitration, and viewing and listening of 
broadcast, and was interrupted, and it is suitable. 
[0002] 

[Description of the Prior Art] As an approach of recording digital television 
broadcasting, the conventional example is shown in drawing 8 . every of the 
program which drawing 8 (a) changed into the MPEG 2 transport stream 
(henceforth referred to as TS) the digital-broadcasting signal received with the 
receiving demodulator 1, and was chosen with the judgment vessel 2 - it 
classifies to PES and restores to an image, voice, and data to the signal 
corresponding to a drop 4 with the MPEG demodulator 3. 
[0003] Generally, record is the approach of being changed into analog video and 
an audio signal, using the video tape recorder 101 of analogs, such as VHS, and 
recording on a magnetic tape. Moreover, playback inputs the recorded analog 
video and O 1 DIO signal into a drop. 



[0004] Moreover, drawing 8 (b) changes into TS the digital-broadcasting signal 
received with the receiving demodulator 1, and changes it into TS of only the 
program chosen with the judgment vessel 2. And it is the approach of using the 
video tape recorder 102 which can record TS, such as D-VHS, for TS data which 
carried out [ above-mentioned ] conversion on a magnetic tape. 
[0005] Playback returns recorded TS data to the judgment machine 2, classifies 
to each PES and restores to an image, voice, and data to the signal 
corresponding to a drop 4 with the MPEG demodulator 3. These approaches 
record time amount broadcast of arbitration, and rewind and reproduce it after 
record termination. 

[0006] Although the magnetic tape is suitable for saving large capacity data, it 
does not fit random access. Magnetic tapes were the objects with sequential 
main actuation on the property. 

[0007] So, in order to perform random actuation, the activity of semiconductor 
memory, a magnetic disk (HD), a magneto-optic disk (MO), an optical disk (DVD, 
CD), etc. can be considered. Furthermore, in order to realize large capacity, as 
realistic media, disc mold record regenerative apparatus, such as HD, MO, and 
DVD, are common. 



[0008] 

[Problem(s) to be Solved by the Invention] However, although random access 
was completed at the high speed, when record and playback occur 
simultaneously, migration of the head which is the record playback section of 
data will occur, and the time amount which performs actual record/playback will 
become late. 

[0009] Since the data rate (bit rate) of digital broadcasting was usually the about 
6 Mbps need in broadcast, the bit rate which a record regenerative apparatus 
must realize also actually had the problem for which the value beyond this is 
needed. 

[0010] This invention aims at the thing which embraced the class of broadcast 
which he is watching in the system reproduced while recording digital television 
broadcasting and for which catch up and it enables it to realize playback in view 
of an above-mentioned trouble. 
[0011] 

[Means for Solving the Problem] The image and voice to which the delay 
regenerative apparatus of this invention is sent with a digital signal, and other 
information supply means to supply the multiplexing data information of data, 



The record playback means for recording the bit stream continuously outputted 
from the above-mentioned information supply means, and reproducing, A data 
control means to perform the entry-of-data control recorded on the 
above-mentioned record playback means, and the output control of the data 
reproduced from the above-mentioned record playback means, It is 
characterized by providing a performance monitor means to supervise a motion 
of the image part of the data recorded on the above-mentioned record playback 
means, and a frame number decision means to determine the frame number 
reproduced from the above-mentioned record playback means to unit time 
amount based on the monitor result of the above-mentioned performance 
monitor means. Moreover, the place by which it is characterized [ of the delay 
regenerative apparatus of this invention / other ] is characterized by for the 
monitor result of the above-mentioned performance monitor means asking for 
the sum of the motion vector of a macro block, and generating it based on the 
magnitude. Moreover, the place by which it is characterized [ of others of the 
delay regenerative apparatus of this invention ] is characterized by for the 
monitor result of the above-mentioned performance monitor means asking for 
the sum of the absolute value of the motion vector of a macro block, and 



generating it based on the magnitude. Moreover, the above-mentioned macro 
block is extracted only from several places of the screen, and with the 
combination of the sum of a motion vector, and the sum of the absolute value of 
a motion vector, the place by which it is characterized [ of others of the delay 
regenerative apparatus of this invention ] patternizes a motion of an image, and 
is characterized by considering as the output of the above-mentioned 
performance monitor means. Moreover, the place by which it is characterized 
[ of others of the delay regenerative apparatus of this invention ] Images, such as 
broadcast of a digital signal, voice, and other information supply means to supply 
the multiplexing data information of data, The record playback means for 
recording the bit stream continuously outputted from the above-mentioned 
information supply means, and reproducing, The entry-of-data control recorded 
on the above-mentioned record playback means, and a data control means to 
control the output of the data reproduced from the above-mentioned record 
playback means, It is characterized by providing a sound-volume monitor means 
to supervise the sound volume of the voice part of the data recorded on the 
above-mentioned record playback means, and a frame number decision means 
to determine the frame number reproduced from the above-mentioned record 



playback means to unit time amount based on the monitor result of the 
above-mentioned sound-volume monitor means. Moreover, the place by which it 
is characterized [ of others of the delay regenerative apparatus of this invention ] 
is characterized by providing the image storage means for compounding a 
display image, and an expansion composition means to develop and compound 
the frame of the frame number reproduced to unit time amount for the 
above-mentioned image storage means from the above-mentioned record 
playback means. 

[0012] The image and voice to which the delay playback approach of this 
invention is sent with a digital signal, and the other information provisioning 
processes which supply the multiplexing data information of data, The record 
regeneration which records the bit stream continuously outputted by the 
above-mentioned information provisioning process on a record playback means, 
and is reproduced, The data control processing which performs the entry-of-data 
control recorded on the above-mentioned record playback means, and the 
output control of the data reproduced with the above-mentioned record playback 
means, It is characterized by performing performance monitor processing which 
supervises a motion of the image part of the data recorded on the 



above-mentioned record playback means, and frame number decision 
processing in which the frame number reproduced to unit time amount with the 
above-mentioned record playback means is determined based on the output of 
the above-mentioned performance monitor processing. Moreover, the place by 
which it is characterized [ of the delay playback approach of this invention / 
other ] is characterized by for the output of the above-mentioned performance 
monitor processing asking for the sum of the motion vector of a macro block, and 
generating it based on the magnitude. Moreover, the place by which it is 
characterized [ of others of the delay playback approach of this invention ] is 
characterized by for the output of the above-mentioned performance monitor 
processing asking for the sum of the absolute value of the motion vector of a 
macro block, and generating it based on the magnitude. Moreover, the 
above-mentioned macro block is extracted only from several places of the 
screen, and with the combination of the sum of a motion vector, and the sum of 
the absolute value of a motion vector, the place by which it is characterized [ of 
others of the delay playback approach of this invention ] patternizes a motion of 
an image, and is characterized by considering as the output of the 
above-mentioned performance monitor processing. Moreover, the place by 



which it is characterized [ of others of the delay playback approach of this 
invention ] Images, such as broadcast of a digital signal, voice, and the other 
information provisioning processes that supply the multiplexing data information 
of data, The record regeneration which records the bit stream continuously 
outputted by the above-mentioned information provisioning process on a record 
playback means, and is reproduced, The entry-of-data control recorded on the 
above-mentioned record playback means, and the data control processing 
which controls the output of the data reproduced with the above-mentioned 
record playback means, It is characterized by performing sound-volume monitor 
processing which supervises the sound volume of the voice part of the data 
recorded on the above-mentioned record playback means, and frame number 
decision processing in which the frame number reproduced to unit time amount 
with the above-mentioned record playback means is determined based on the 
output of the above-mentioned sound-volume monitor processing. Moreover, the 
place by which it is characterized [ of others of the delay playback approach of 
this invention ] is carrying out performing image storage processing for 
compounding a display image, and expansion composition processing which 
develops and compounds the frame of the frame number reproduced to unit time 



amount with the above-mentioned record playback means to the 
above-mentioned image storage processing as the description. 
[0013] The storage of this invention is characterized by storing from a computer 
the program for operating a computer as each means to constitute the 
above-mentioned delay regenerative apparatus possible [ read-out ]. Moreover, 
the place by which it is characterized [ of the storage of this invention / other ] is 
characterized by storing the program for making a computer perform the 
procedure of the above-mentioned delay playback approach possible [ read-out] 
from a computer. 
[0014] 

[Embodiment of the Invention] [Gestalt of the 1st operation] The gestalt of one 
operation of the delay regenerative apparatus of this invention, the delay 
playback approach, and a storage is hereafter explained using a drawing. 
Drawing 1 is system configuration drawing in this gestalt, and 1 is a receiving 
demodulator which receives digital-broadcasting signals, such as a satellite, 
CATV, and a ground wave, and outputs the transport stream (henceforth 
referred to as TS) of MPEG 2. 

[0015] The MPEG demodulator changed into the signal which the judgment 



machine which classifies TS by which 2 is constituted from two or more 
programs to each PAKETTAIZUDO elementary stream (henceforth referred to 
as PES), and 3 restore to each PES to an image, voice, and data, and can be 
displayed on a drop, and 4 display the video signal to which it restored, and are 
the drop of a ****** sake about a sound signal. 

[0016] 5 is a data control machine, while changing broadcast, data logging, and 
data playback with the directions from a microcomputer 10, only TS data of one 
selected program are classified in the case of data logging, and TS data are 
outputted in accordance with the threshold of the reproduction speed directed 
with a microcomputer 10 based on the classification information which the frame 
sorter 6 added in the case of data playback. 

[0017] It is a frame sorter, 6 is calculated on each frame (field) of every, 
classifies a motion into four patterns, divides the amount of motions into eight 
steps, and in case it records TS data on the data storage 9, it adds this 
classification information. 

[0018] 7 is a buffer and R/W more nearly high-speed than the data storage 9 for 
saving TS data recorded on the data storage 9 temporarily is possible for it. 8 is 
a buffer for saving TS data reproduced from the data storage 9 temporarily, and 



R/W more nearly high-speed than the data storage 9 is possible for it. 
[0019] Data storage for 9 to record TS data and 10 are microcomputers which 
control the data control machine 5 and a control indicator 1 1 by the directions 
from the outside. The control indicator which compounds 11 in the memory 12 
for image composition according to the reproduction speed to which the data 
control machine 5 outputs the frame image to which it restored with the MPEG 
demodulator 3, and controls an output, and 12 are the memory for image 
composition for compounding the image according to reproduction speed. 
[0020] It gets over with the receiving demodulator 1 which suited various 
transmission lines, and the digital-broadcasting signal which was multiplexed by 
the MPEG 2 method and was able to add various modulations is changed into 
TS of MPEG 2. As for this TS, it is common that two or more programs are 
multiplexed. 

[0021] In case it views and listens to broadcast, the data control machine 5 
classifies TS of the single program chosen as arbitration in the judgment vessel 
2 at delivery and each PES, and restores to it to an image, voice, and data with 
the MPEG demodulator 3, delivery and an image are displayed on an indicator 4, 
voice is reproduced, and data are processed according to an application. In 



order to perform delay playback, in case broadcast is recorded on the data 
storage 9, it is carried out in inputting the command which records on a 
microcomputer 10. 

[0022] A microcomputer 10 will be ordered to record TS data on the data control 
machine 5, if a record command is received. The data control machine 5 stops 
sending TS data for TS data of the single program chosen as arbitration to the 
frame sorter 6 to delivery and the judgment machine 2. 

[0023] The flow chart of TS data-logging actuation is shown in drawing 2 . In the 
first step S21, a microcomputer 10 calculates the classification information on an 
image, next adds the above-mentioned classification information in step S22, 
and sends TS data to a buffer. 

[0024] Next, in step S23, it records on data storage. Next, at step S24, record 
judges whether it is a termination, and when it is not a termination, it is 
performed by repeating the actuation returned and mentioned above to step S21. 
Moreover, in a termination, record actuation is ended as a result of decision of 
step S24. 

[0025] Count of a motion vector performed by the frame sorter 6 is performed for 
the specific macro block in an image. A vector is classified into four patterns [ for 



the macro block count individual in five in a screen ], and it moves according to 
each pattern, an amount is calculated, and it classifies into eight steps according 
to the gestalt of this operation. 

[0026] Four patterns are shown in drawing 3 . (1) - intra it is a coded image 
and a motion vector cannot be extracted. For (2), all motion vectors are the 
same direction. For (3), all motion vectors are sense about one point. (4) does 
not have correlation in all motion vectors. 

[0027] It classifies into the above four patterns, and not according to a vector 
sum or the scalar sum but according to a pattern, it calculates and moves simply, 
an amount is divided into eight steps, and it considers as the classification 
information on an image. This information is added and TS data are recorded on 
the data storage 9. 

[0028] This actuation is repeated until a microcomputer 10 outputs a record quit 
command. When reproducing recorded TS data from the data storage 9, it is 
carried out in inputting the command reproduced on a microcomputer 10. 
[0029] Playback can be brought forward for reproduction speed, in order to catch 
up with real broadcast. A rate can choose one of the approaches to which a rate 
is changed based on the classification information added at the time of constant 



speed and record. In the case of constant speed, the reproduction speed L (L= 1 , 
2, 3 ....) set as arbitration performs **** playback. TS data are classified by one L 
times the bit rate [ delivery and ] of this through the data control machine 5 at 
delivery and each PES at the judgment machine 2 to a buffer 8, and it gets over 
to an image, voice, and data with the MPEG demodulator 3. 
[0030] The example of TS data playback actuation is shown in drawing 4 . In the 
first step S41, it judges whether it is constant **** playback. As a result of this 
decision, in not being constant **** playback, it progresses to step S42, and 
processing set to reproduction speed weight =m (m is an integer) is performed. 
[0031] Next, in step S43, processing set to reproduction speed =M (M is an 
integer) by classification information and reproduction speed weight m is 
performed. Next, in step S44, processing which compounds the image of M 
sheets in the image of one sheet is performed. 

[0032] Next, in step S45, voice is reproduced by MX. Then, playback judges 
whether it is a termination at step S46, and, in a termination, playback is ended. 
Moreover, in continuing playback, the processing returned and mentioned above 
to step S41 is repeated, and it performs it. 

[0033] On the other hand, as a result of decision of step S41, in constant **** 



playback, it progresses at step S47, and processing set to reproduction speed 
=L (L is an integer) is performed. Then, it progresses to step S48, processing 
which compounds the image of L sheets to one sheet is performed, and, 
subsequently voice is reproduced by LX in step S49. Then, playback judges 
whether it is a termination at step S46. 

[0034] In the case of constant speed, the playback approach of an image 
performs **** playback with the reproduction speed L (L= integer) set as 
arbitration. TS data are classified by one L times the bit rate [ delivery and ] of 
this through the data control machine 5 at delivery and each PES at the 
judgment machine 2 to a buffer 8, and it gets over to an image, voice, and data 
with the MPEG demodulator 3. 

[0035] The composition at the time of 3X is shown in drawing 5 . The image 
decoded by the frame buffer is equally divided into three perpendicularly, and a 
part for a display is copied to the memory 12 for image composition. Thus, the 
compounded image is returned to the MPEG demodulator 3, a delivery image is 
displayed on an indicator 4, voice is reproduced, and data are processed 
according to an application. A reproduced flag is added to reproduced TS data in 
that case. 



[0036] In the playback using classification information, the catching-up playback 
which reproduces a part without motions, such as a sport relay broadcast, at a 
high speed is realizable. The approach of reproducing at a high speed 
determines reproduction speed M based on the integer m and eight steps of 
classification information which were set as arbitration. The processing after this 
is equivalent to the case of constant speed. 

[0037] As mentioned above, when continuing viewing and listening of the 
broadcast interrupted temporarily, you can understand the content by adjusting 
reproduction speed according to a motion also in a scene which has many 
motions. Moreover, since the catching-up rate of catching-up playback can be 
adjusted, doubling with liking of a user is also easy. 

[0038] [Gestalt of the 2nd operation] The gestalt of operation of the 2nd of this 
invention is hereafter explained using a drawing. In addition to the amount of 
motions of an image, in the delay playback system of this gestalt, it is 
characterized by the ability to adjust reproduction speed also with audio sound 
volume. The difference from the gestalt of the 1st operation is an image and 
voice generating eight steps of classification information, and adding to TS data 
in the frame sorter 6. System configuration drawing is the same as that of the 



gestalt of the 1st operation. 

[0039] The frame sorter 6 calculates by classifying a motion of each frame (field) 
into four patterns, moves, and divides an amount into eight steps. Moreover, in 
case audio magnitude is divided into eight steps and TS data are recorded on 
the data storage 9, the above-mentioned classification information is added. It 
gets over with the receiving demodulator 1 which suited various transmission 
lines, and the digital-broadcasting signal which was multiplexed by the MPEG 2 
method and was able to add various modulations is changed into TS of MPEG 2. 
As for this TS, it is common that two or more programs are multiplexed. 
[0040] In case it views and listens to broadcast, the data control machine 5 TS of 
the single program chosen as arbitration in the judgment vessel 2 Delivery, 
Classify to each PES and it gets over to an image, voice, and data with the 
MPEG demodulator 3. In order to perform delay playback by which delivery and 
an image are displayed on an indicator 4, voice is reproduced, and data are 
processed according to an application, in case broadcast is recorded on the data 
storage 9, it is carried out in inputting the command which records on a 
microcomputer 10. 

[0041] A microcomputer 10 will be ordered to record TS data on the data control 



machine 5 if a record command is received. The data control machine 5 stops 
sending TS data for TS data of the single program chosen as arbitration to the 
frame sorter 6 to delivery and the judgment machine 2. 

[0042] Next, a motion vector is calculated by the frame sorter 6. (Refer to the 
gestalt of the 1st operation) Further, the frame sorter 6 divides audio magnitude 
into eight steps, and is taken as the classification information on audio. Such 
information is added and TS data are recorded on the data storage 9. This 
actuation is repeated until a microcomputer 10 outputs a record quit command. 
In case recorded TS data are reproduced from the data storage 9, it is carried 
out in inputting the command reproduced on a microcomputer 10. 
[0043] The example of TS data-logging actuation is shown in drawing 6 . In the 
first step S61, a microcomputer 10 calculates an image and the classification 
information on audio, next adds the above-mentioned classification information 
in step S62, and sends TS data to a buffer. 

[0044] Next, in step S63, it records on data storage. Next, at step S64, record 
judges whether it is a termination, and when it is not a termination, it is 
performed by repeating the actuation returned and mentioned above to step S61 . 
Moreover, in a termination, record actuation is ended as a result of decision of 



step S64. 

[0045] Playback can be brought forward for reproduction speed, in order to catch 
up with real broadcast. A rate can choose one of the approaches to which a rate 
is changed based on the classification information added at the time of constant 
speed and record. In the playback using classification information, it separates 
to image serious consideration or voice serious consideration further. It can 
choose according to the class of broadcast to which it views and listens. 
[0046] With the algorithm of the gestalt of this operation, a sport relay broadcast 
etc. can realize catching-up playback which reproduces a part without a motion 
at a high speed. However, little broadcast of a motion like an orchestra concert 
will be dramatically reproduced at high speed, if it reproduces by image serious 
consideration. 

[0047] Since the object is for such a music program to listen to music on an 
image, the place (small place) out of which the sound has not come is 
reproduced and flown at a high speed, and if a performance starts, the 
catching-up playback reproduced ordinarily is realizable. The approach of 
reproducing at a high speed determines reproduction speed N (or M) based on 
the integer m and eight steps of classification information which were set as 



arbitration. The processing after this is equivalent to the case of constant speed. 
[0048] The example of TS data playback actuation is shown in drawing 7 . In the 
first step S71, it judges whether it is constant **** playback. As a result of this 
decision, in not being constant **** playback, it progresses to step S72, and 
processing set to reproduction speed weight =m (m is an integer) is performed. 
[0049] Next, in step S73, it judges whether it is playback of image serious 
consideration. As a result of this decision, in not being playback of image serious 
consideration, it progresses to step S74, and classification information and 
reproduction speed weight m perform processing set to reproduction speed =N 
(N is an integer). Next, in step S75, processing which compounds the image of N 
sheets in the image of one sheet is performed. 

[0050] Next, in step S76, voice is reproduced by MX. Then, playback judges 
whether it is a termination at step S83, and, in a termination, playback is ended. 
Moreover, in continuing playback, the processing returned and mentioned above 
to step S71 is repeated, and it performs it. 

[0051] On the other hand, as a result of decision of step S71, in constant **** 
playback, it progresses at step S77, and processing set to reproduction speed 
=L (L is an integer) is performed. Then, it progresses to step S78, processing 



which compounds the image of L sheets to one sheet is performed, and, 
subsequently voice is reproduced by LX in step S79. Then, playback judges 
whether it is a termination at step S83. 

[0052] Furthermore, in step S73, when it is judged that it is playback of image 
serious consideration, it progresses to step S80 and processing set to 
reproduction speed =M (M is an integer) by classification information and 
reproduction speed weight m is performed. Next, in step S81 , processing which 
compounds the image of M sheets in the image of one sheet is performed. 
[0053] Next, in step S82, voice is reproduced by MX. Then, playback judges 
whether it is a termination at step S83, and, in a termination, playback is ended. 
Moreover, in continuing playback, the processing returned and mentioned above 
to step S71 is repeated, and it performs it. 

[0054] As mentioned above, when continuing viewing and listening of the 
broadcast interrupted temporarily, it can continue by adjusting reproduction 
speed according to the class of broadcast, without spoiling the object of viewing 
and listening. Moreover, since the catching-up rate of catching-up playback can 
be adjusted, doubling with liking of a user is also easy. 

[0055] (Other operation gestalten of this invention) Even if it applies this 



invention to the system which consists of two or more devices (for example, a 
host computer, an interface device, a reader, a printer, etc.), it may be applied to 
the equipment which consists of one device. 

[0056] Moreover, so that the function of the operation gestalt mentioned above 
may be realized and various kinds of devices may be operated As opposed to 
the computer in the equipment connected with the various above-mentioned 
devices, or a system The program code of the software for realizing the function 
of the above-mentioned operation gestalt is supplied. What was carried out by 
operating the various above-mentioned devices according to the program stored 
in the computer (CPU or MPU) of the system or equipment is contained under 
the category of this invention. 

[0057] Moreover, the function of the operation gestalt which the program code of 
the above-mentioned software itself mentioned above in this case will be 
realized, and the storage which stored the means for supplying that program 
code itself and its program code to a computer, for example, this program code, 
constitutes this invention. As a storage which memorizes this program code, a 
floppy disk, a hard disk, an optical disk, a magneto-optic disk, CD-ROM, a 
magnetic tape, the memory card of a non-volatile, ROM, etc. can be used, for 



example. 

[0058] Moreover, by performing the program code with which the computer was 
supplied, also when [, such as OS (operating system) or other application 
software with which the function of an above-mentioned operation gestalt is not 
only realized, but the program code is working in a computer, ] the function of an 
above-mentioned operation gestalt is realized jointly, it cannot be 
overemphasized that this program code is contained in the operation gestalt of 
this invention. 

[0059] Furthermore, after stored in the memory with which the functional 
expansion unit by which the supplied program code was connected to the 
functional add-in board and the computer of a computer is equipped, also when 
the function of the operation gestalt which the CPU with which the functional 
add-in board and functional expansion unit are equipped based on directions of 
the program code performed a part or all of actual processing, and mentioned 
above by the processing is realized, it cannot be overemphasized that it is 
contained in this invention. 
[0060] 

[Effect of the Invention] As explained above, according to this invention, in the 



system reproduced while recording digital television broadcasting, the 
catching-up playback according to the class of broadcast which he is watching is 
realizable. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is system configuration drawing in the gestalt of operation of this 
invention. 

[Drawing 2] It is the flow chart of TS data logging. 

[Drawing 3] It is drawing showing a motion vector extract pattern. 

[Drawing 4] It is the flow chart of TS data playback. 

[Drawing 5] It is drawing showing the example of image composition. 

[Drawing 6] It is the flow chart of TS data logging. 

[Drawing 7] It is the flow chart of TS data playback. 

[Drawing 8] It is system configuration drawing showing an example of the 
conventional technique. 
[Description of Notations] 

1 Receiving Demodulator 

2 Judgment Machine 

3 MPEG Demodulator 

4 Drop 

5 Data Control Machine 

6 Frame Sorter 



7 Buffer 

8 Buffer 

9 Data Storage 

10 Microcomputer 

11 Control Indicator 

12 Memory for Image Composition 
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